Automatic program formatting for tv displays

ABSTRACT

A smart multimedia content receiver automatically resizes video images based on the content being displayed on the TV screen. Such a self-formatting content receiver includes on-board image processing capability that provides continuous video analysis to detect changes in program formatting and convert each frame as it is received in real time, as opposed to processing and re-releasing an entire program or movie, or relying on the viewer to re-format programs manually. In response to detecting a change, aspect ratio adjustments are made as needed. Because the self-formatting content receiver has access to the video data before it is displayed, such automatic on-the-fly adjustments ensure that the viewer&#39;s experience during program changes is seamless and without distortion. Subscribers can influence decisions made by the content receiver by pre-setting viewer preferences for aspect ratio adjustment.

BACKGROUND Technical Field

The present disclosure generally relates to TV displays, and inparticular, to automatically formatting media content for display on aTV screen.

Description of the Related Art

Movies and television programming use various image size formats forprojecting images onto movie screens or television displays. Formatsthat have been used historically originated in the film industry andwere based on the size of individual frames used in motion picture film,e.g., 28 mm, 35 mm, and so on. The high-definition format currently inuse for broadcast television in the U.S. (HDTV) has an aspect ratio of16:9, meaning the picture is rectangular, having dimensions of 16 unitswide and 9 units high. The 16:9 aspect ratio is equivalently expressedas the normalized ratio 1.77:1. Other aspect ratios commonly used forvideo include widescreen TV format having an aspect ratio of 4:3,equivalent to 1.33:1, various intermediate aspect ratios between 1.33:1and 1.77:1, 1.85:1, and 2.4:1. Older TV shows that were originallyfilmed for widescreen TV, and older movies, can be re-mastered by aproduction studio to convert the format to the 1.77:1 format.

With the recent advent of many types of displays, the video content maynot have a standard aspect ratio of a type commonly used. For example,videos are being recorded on many different brands of cell phones, videocameras, and various portable electronic devices. With the wideavailability of many types of video recording devices and video cameras,the aspect ratio of a particular recorded video cannot be easilydetermined in advanced, and may vary over a wide range. In addition, alarge number of different displays are now available. Displays can be assmall as a watch or a cell phone, and as large as the size of a wall.Further, the aspect ratio of a particular display may not fit anystandard, and may vary widely from one display to another.

The term high definition (HD), as it is used today, refers to a highdensity video image having a resolution of at least 1280×720 pixels,projected onto a display at a frame rate of 60 frames/sec, or 60 Hz.More specifically, HD video has 720 horizontal lines and 1280 verticalcolumns, wherein 1280/720=16:9=1.77. While a standard SD-DVD disc storesup to 4.5 GB of standard resolution video image data, a typical HD-DVDvideo disc stores up to 15 GB of HD video image data.

TV displays typically allow viewers to configure the screen differentlyas needed for different resolution programs having different aspectratios. Depending on the screen size and the format of the program, thevideo images may fill the entire screen, or there may be black verticalor horizontal sidebars at the edges of the screen, filling in the unusedportion, or a portion of the image may bleed off the screen and not bevisible. The viewer may have to decide whether to make the image sizesmaller than the screen and tolerate the sidebars, or lose some of thepicture because it is too big to fit on the screen. Once the vieweradjusts the screen according to the desired format for the currentprogram, if the viewer switches channels or if a commercial comes on,the adjustment made for the initial program may not be suitable for thenew video images. For example, commercials may be distorted or a newshow may be partially off the screen, thus requiring the viewer tomanually re-adjust the format settings every time they switch from oneprogram to another. Such constant re-adjustment is a nuisance for theviewer.

In the case of commercials in particular, it is unlikely that a viewerwill bother to manually adjust the format settings for each newcommercial that is broadcast. As a result, commercial advertisingmessages may be lost on the audience because viewers are distracted bydistorted commercial images. Alternatively, a portion of the commercialmessage that exceeds the size of the screen may not be visible, whichcould impair its effectiveness in marketing to the viewer.

BRIEF SUMMARY

Due to a high level of competition among entertainment contentproviders, those who are able to eliminate video formatting problems maygain a business advantage when competing for subscribers andadvertisers. Such an advantage may be attained through the use of asmart content receiver, or smart set top box (STB), that automaticallyresizes video images based on the content being displayed on the TVscreen. Such a self-formatting STB includes on-board image processingcapability that provides continuous video analysis to detect changes inprogram formatting and convert each frame as it is received in realtime, as opposed to processing and re-releasing an entire program ormovie, or relying on the viewer to re-format programs manually. Inresponse to detecting a change, the self-formatting STB makes aspectratio adjustments as needed. Because the self-formatting STB has accessto the video data before it is displayed, such automatic on-the-flyadjustments ensure that the viewer's experience during program changesis seamless and without distortion. Subscribers can influence decisionsmade by the STB by pre-setting viewer preferences for aspect ratioadjustment.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings, identical reference numbers identify similar elements.The sizes and relative positions of elements in the drawings are notnecessarily drawn to scale.

FIG. 1 is a schematic illustration of an exemplary residential mediaentertainment system that provides automatic program formatting,according to one embodiment described herein.

FIG. 2 is a block diagram showing components of a content receiver thatincludes an automatic program formatting feature, according to anembodiment described herein.

FIG. 3 is a high-level flow diagram summarizing flow of multimediainformation in a system that supports automatic program formatting,according to one embodiment described herein.

FIGS. 4A and 4B are pictorial views showing exemplary input and outputformats that a viewer would see when using an automatic programformatting feature to transform high definition video content asdescribed herein.

FIG. 5 is a flow diagram showing a sequence of steps in acomputer-implemented method of automatic program formatting, accordingto a first embodiment as described herein.

FIG. 6A is a pictorial view of an exemplary unformatted scene.

FIGS. 6B-6D are pictorial views of the exemplary scene of FIG. 6Aformatted according to several different aspect ratios, as describedherein.

FIG. 7A is a schematic view showing a first method of displayingadjacent pixels from the scene shown in FIG. 6A on a higher resolutiondisplay, according to one embodiment described herein.

FIG. 7B is a schematic view showing a second method of displayingadjacent pixels from the scene shown in FIG. 6A on a higher resolutiondisplay, according to one embodiment described herein.

FIG. 8 is a flow diagram showing a sequence of steps in acomputer-implemented method of automatic program formatting, accordingto a second embodiment as described herein.

DETAILED DESCRIPTION

In the following description, certain specific details are set forth inorder to provide a thorough understanding of various aspects of thedisclosed subject matter. However, the disclosed subject matter may bepracticed without these specific details. In some instances, well-knownstructures and methods of video display, comprising embodiments of thesubject matter disclosed herein, have not been described in detail toavoid obscuring the descriptions of other aspects of the presentdisclosure.

Unless the context requires otherwise, throughout the specification andclaims that follow, the word “comprise” and variations thereof, such as“comprises” and “comprising” are to be construed in an open, inclusivesense, that is, as “including, but not limited to.”

Reference throughout the specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment. Thus, the appearance of the phrases “in oneembodiment” or “in an embodiment” in various places throughout thespecification are not necessarily all referring to the same aspect.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or more aspects of the presentdisclosure.

In this specification, embodiments of the present disclosure illustratea subscriber satellite television service as an example. This detaileddescription is not meant to limit the disclosure to any specificembodiment. The present disclosure is equally applicable to cabletelevision systems, broadcast television systems, Internet streamingmedia systems, or other television or video distribution systems thatinclude user hardware, typically in the form of a content receiver orset top box that is supported by the media provider or by a third partymaintenance service provider. Such hardware can also include, forexample, digital video recorder (DVR) devices and/or digital-video-disc(DVD) recording devices or other accessory devices inside, or separatefrom, the set top box.

Throughout the specification, the term “subscriber” refers to an enduser who is a customer of a media service provider and who has anaccount associated with the media service provider. Subscriber equipmentresides at the subscriber's address. The terms “user” and “viewer” referto anyone using part or all of the home entertainment system componentsdescribed herein.

The disclosure uses the term “signal” in various places. One skilled inthe art will recognize that the signal can be any digital or analogsignal. Those signals can include, but are not limited to, a bit, aspecified set of bits, an A/C signal, or a D/C signal. Uses of the term“signal” in the description can include any of these differentinterpretations. It will also be understood by one skilled in the artthat the term “connected” is not limited to a physical connection butcan refer to any means of communicatively or operatively coupling twodevices.

As a general matter, the disclosure uses the terms “televisionconverter,” “receiver,” “set top box,” “television receiving device,”“television receiver,” “television recording device,” “satellite set topbox,” “satellite receiver,” “cable set top box,” “cable receiver,” and“content receiver,” to refer interchangeably to a converter device orelectronic equipment that has the capacity to acquire, process anddistribute one or more television signals transmitted by broadcast,cable, telephone or satellite distributors. DVR and “personal videorecorder (PVR)” refer interchangeably to devices that can record andplay back television signals and that can implement playback functionsincluding, but not limited to, play, fast-forward, rewind, and pause. Asset forth in this specification and the figures pertaining thereto, DVRand PVR functionality or devices can be combined with a televisionconverter. The signals transmitted by these broadcast, cable, telephone,satellite, or other distributors can include, individually or in anycombination, Internet, radio, television or telephonic data, andstreaming media. One skilled in the art will recognize that a televisionconverter device can be implemented, for example, as an externalself-enclosed unit, a plurality of external self-enclosed units or as aninternal unit housed within a television. One skilled in the art willfurther recognize that the present disclosure can apply to analog ordigital satellite set top boxes.

As yet another general matter, it will be understood by one skilled inthe art that the term “television” refers to a television set or videodisplay that can contain an integrated television converter device, forexample, an internal cable-ready television tuner housed inside atelevision or, alternatively, that is connected to an externaltelevision converter device such as an external set top box connectedvia cabling to a television. A further example of an external televisionconverter device is the EchoStar Hopper combination satellite set topbox and DVR.

A display may include, but is not limited to: a television display, amonitor display, an interlaced video display, a non-interlaced videodisplay, phase alternate line (PAL) display, National Television SystemCommittee (NTSC) systems display, a progressive scan display, a plasmadisplay, a liquid crystal display (LCD) display, a cathode ray tube(CRT) display and various High Definition (HD) displays, an IMAX™screen, a movie screen, a projector screen, etc.

Specific embodiments are described herein with reference toentertainment systems having automatic features that have been produced;however, the present disclosure and the reference to certain materials,dimensions, and the details and ordering of processing steps areexemplary and should not be limited to those shown.

Turning now to the drawings, FIG. 1 shows a home entertainment system200 that features automatic program formatting. The entertainment system200 includes a content receiver 202, a media presentation device 204having a display 206, and a remote control 208. The content receiver202, e.g., a television set top box (STB), can also be coupled to themedia presentation device 204, or the content receiver 202 can be in theform of hardware built into the media presentation device 204.Instructions that carry out features of the entertainment system 200 arestored in, and/or executed by, components of the content receiver 202.

The content receiver 202 is communicatively coupled to one or moresources of media content 210 (two shown, 210A and 210B) to receive themedia content for presentation via the media presentation device 204.Sources of media content 210 can include one or more of a terrestrialtelevision or radio antenna, a satellite receiving antenna 210A, abroadband cable subscriber service, streaming media received from theInternet 210B directly or via a wireless Internet router, and the like.Media content is provided as a media signal, via communication links214, e.g., a satellite antenna communication link 214A, or an Internetcommunication link 214B. The communication links 214 can accommodate awired signal or a wireless signal. Furthermore, a content provider mayprovide to a viewer 216 multiple media signals, e.g., via satelliteand/or via the Internet as shown, via broadband cable and/or Internet,or in any other suitable way.

The content receiver 202 may be communicatively coupled to one or moreperipheral devices 220 such as a media playback device including, butnot limited to, a DVD player, a VCR, a stereo, a television, a gameconsole, or a stand-alone DVR device that provides an alternate sourceof content directly to the display 206 via a wireless or a wiredconnection 222. Any of the peripheral devices 220 can besubscriber-owned devices, or they can be supplied by a mediasubscription service.

The satellite receiving antenna 210A receives media content via asatellite signal 211 from one or more satellite transponders in earthorbit. Each satellite transponder is, for purposes of the entertainmentsystem 200, a source of content that transmits one or more mediachannels, such as HBO, ESPN, pay-per-view channels, etc., to thesatellite receiving antenna 210A. A satellite television distributor cantransmit one or more satellite television signals to one or moresatellites. Satellite television distributors can utilize severalsatellites to relay the satellite television signals to subscribers.Each satellite can have several transponders. Transponders transmit thesatellite signal from the satellite to the satellite receiving antenna210A.

The media presentation device 204 having the display 206 can be anyelectronic device that presents media content to a user; for example, atelevision, a radio, a computer, a mobile computing device such as alaptop, a tablet, a gaming console, a smart phone, or the like, or thedisplay 206 itself. The display 206 can be any kind of video displaydevice, such as a cathode ray tube display, a liquid crystal display(LCD), a plasma display, a television, a computer monitor, a rearprojection screen, a front projection screen, a heads-up display, or anyother electronic display device. The display 206 can be separate from,or integrated into, the media presentation device 204. The display 206can include audio speakers, or the display 206 can be coupled toseparate audio speakers. The term “for display” as used herein generallyincludes presentation of an audio component as well as a video componentof the media signal.

The remote control 208 is configured to communicate with the contentreceiver 202 via a wireless connection path 212, for example, aninfrared (IR) signal. The remote control 208 can be operated by theviewer 216 to cause the content receiver 202 to display received contenton the media presentation device 204. The remote control 208 may also beused to display a programming guide on the display 206 and tocommunicate program selections to the content receiver 202. The remotecontrol 208 can also be used to send commands to the content receiver202, including channel selections, display settings, format selections,and the like. The wireless connection path 212 can use, for example,infrared or UHF transmitters within the remote control 208. One exampleof an embodiment of the remote control 208 is the EchoStar TechnologiesCorporation 40.0 Remote Control that includes an IR transmitter and anultra-high frequency (UHF) transmitter. The remote control 208 may beable to send signals to other peripheral devices that form part of theentertainment system 200. The content receiver 202 may also be able tosend signals to the remote control 208, including, but not limited to,signals to configure the remote control 208 to operate other peripheraldevices in the entertainment system 200. In some embodiments, the remotecontrol 208 has a set of Light Emitting Diodes (LEDs). Some remotecontrols can include Liquid Crystal Displays (LCDs) or other displayscreens. The remote control 208 can include buttons, dials, or otherman-machine interfaces. While the remote control 208 is often the meansfor a user to communicate with the content receiver 202, one skilled inthe art will recognize that other means of communicating with thecontent receiver 202 are available, including, but not limited toattached keyboards, smart phones, front panel buttons or touch screens.

FIG. 2 shows components of the content receiver 202, according to oneembodiment described herein. The content receiver 202 is a set top boxequipped with instructions that carry out an embodiment of the presentdisclosure. Components of the content receiver 202 include controlcircuitry 302, one or more tuners 304 (two shown, 304 a, 304 b), anoptional internal digital video recorder (DVR) 306, a non-transitorycomputer readable memory (NTCRM) 308, a network interface 310, aperipheral interface 312, and a remote control interface 314. A set ofautomatic program formatting instructions 316 may reside in the memory308 for execution by one or more microprocessors within the controlcircuitry 302, e.g., by a general purpose central processing unit (CPU),or a specialized image processing unit (IPU).

Generally, the content receiver 202 can receive one or more televisionsignals from a media provider such as a cable television distributor, abroadcast television distributor, the Internet, or a satellitetelevision distributor. In addition, in the embodiment shown, theentertainment system 200 receives electronic program guide (EPG)information from a satellite television distributor via the contentreceiver 202. One skilled in the art will recognize that the contentreceiver 202 can also receive video-digital subscriber line (DSL), DSL,Internet, wireless and other signals from content or video distributors.The content receiver 202 can process television signals and can send theprocessed signals to peripheral electronic devices, such as the display206 and the remote control 208. The content receiver 202 also can acceptcommands from the remote control 208 or other peripheral electronicdevices. One skilled in the art will recognize that many embodiments ofthe entertainment system 200 are possible and within the scope of thisdisclosure. Other such embodiments can include, but are not limited to,various combinations or permutations of devices and connections for thedelivery, storage, and display of communications, content and otherdata.

In one embodiment, the content receiver 202 receives media content fromthe satellite receiving antenna 210A. Each tuner 304 tunes into aselected media channel received by the satellite receiving antenna 210Ato acquire the satellite signal 211. Tuners 304 initially process thesatellite signal. When the content receiver 202 includes multiple tuners304, the content receiver 202 can record two or more programs that airsimultaneously. Generally, the content receiver 202 can record one showat a single time for each tuner 304 that the content receiver 202includes. For example, if the content receiver 202 includes two tuners,then the content receiver 202 can record and/or view two media programssimultaneously. Thus if there is a recording conflict between twoprograms, the content receiver 202 can still record both programsbecause it has two tuners. If a recording conflict occurs because thenumber of tuners 304 is less than the number of programs that the userwants to view or record, the control circuitry 302 can be programmed toresolve the conflict by prioritizing certain media content according tothe subscriber's directions, or automatically, based on variouscriteria.

Each tuner 304 transmits an acquired satellite signal to the controlcircuitry 302 where the signal can undergo more extensive signalprocessing. The control circuitry 302 can include a content receiverdecoder, such as, for example, the STi5517 Low-Cost Interactive Set topbox Decoder, Part No. 7424736A, available from STMicroelectronics, Inc.Signals from the control circuitry 302 can include, but are not limitedto, a signal to tune to a transponder as part of the process ofselecting a certain channel for viewing on a peripheral device. A firststep in signal processing by the control circuitry 302 can include, forexample, demodulating the satellite signal. Further signal processingcan include error checking. In one embodiment, the satellite signal 211is in digital form, for example, a digital stream, after demodulationand error correction. The digital stream may use, for example, thedigital video broadcasting (DVB) transport standard. The digital streammay be multiplexed and therefore require de-multiplexing to separate thedigital stream into separate digital data streams. Each of the separatedigital data streams may also be encoded and/or compressed to providethe system with increased bandwidth. In satellite television, encodingformats may include the MPEG, MPEG2 or MPEG4 standards. One skilled inthe art will recognize that systems with analog data or combined analogand digital data are also possible and contemplated herein.

The control circuitry 302 may further include one or more videoprocessing units that, among other video processing operations, maydecode the encoded digital television signal. The video processing unitsmay include, for example, a graphics processor, an MPEG-2 decoder, and adisplay compositor with separate on-screen display (OSD) control forperipheral devices. The control circuitry 302 may further include avideo encoder that encodes a digital stream for output to one or moreperipheral devices, including, but not limited to, a television.Encoding may allow program data to be compressed. The control circuitry302 may also include a storage device interface that couples the controlcircuitry 302 to storage devices such as the DVR 306 and the memory 308.An audio processing unit may also be part of the control circuitry 302.The audio processing unit may, for example, decode the digital streamfor output to peripheral devices, including, but not limited to, astereo, television speakers or portable audio or video players. Theaudio processing unit may include one or more processors, memorycomponents or digital to audio converter (DAC) systems.

The control circuitry 302 includes one or more processors such as ageneral purpose central processing unit (CPU). The CPU may executelogical operations to execute functions of the content receiver 202including, but not limited to, channel selection, system maintenance,recording control, EPG display and control, and functions of theentertainment system 200. Examples of commercially available CPUsinclude the STMicroelectronics Enhanced ST20 32-bit VL-RISC, or IntelXeon or Atom series processors that are designed or adapted for servers.One skilled in the art will recognize that the CPU may be integratedwith memory or other discrete electronic components. The controlcircuitry 302 may further include a custom application-specificintegrated circuit (ASIC) chip, such as from the LSI Logic G11 family,or FPGA, such as from the Altera Stratix™ family. Alternatively, amicrocontroller can be substituted for the CPU. Microcontrollersgenerally include, in addition to a CPU, extra program memory such asread-only memory (ROM), and input/output (I/O) functionality. Thecontrol circuitry 302 may further include one or more memory componentsthat may be used for many purposes, including, but not limited to,storing EPG data and storing data for use by the CPU. Memory componentsmay include volatile memory such as SDRAM memory chips and non-volatilememory devices such as ROM, SRAM, SDRAM and Flash ROM. One skilled inthe art will recognize that volatile memory and non-volatile memory maybe integrated within other electronic components and that other memorycomponents may be included within content receiver 202 and controlcircuitry 302.

The DVR 306 may be used for many purposes, including, but not limitedto, storing recorded programs and buffering currently playing programsto pause or rewind a program. The DVR 306 can be used to record programslocally in the content receiver 202 so that the user may view the TVshow at a later, more convenient time. When the user selects a programto be recorded to the DVR 306, the content receiver 202 sets a recordingtimer that causes the content receiver 202 to automatically record theselected program at the scheduled time. At a time convenient to theuser, the user can operate the remote control 208 to cause the contentreceiver 202 to display a list of programs that have been recorded tothe DVR 306. The user can select the desired program from the list ofrecorded programs and the content receiver 202 will play back theselected program.

Alternatively, the content receiver 202 may access, via the Internet210B, recorded programs stored on a remote storage device. In this case,when the user of the content receiver 202 selects a media program to berecorded, the media program is recorded to remote storage instead of tothe DVR 306 inside the content receiver 202. The user of the contentreceiver 202 can schedule media program recordings by accessing theInternet 210B through a smart phone, a PC, a tablet, a laptop, or othersuitable means. The user can log into an account associated with thecontent receiver 202 and can schedule recordings via the Internet 2106.The media programs can be viewed by connecting the content receiver 202to the Internet 2106 to access the remote storage device.

The content receiver 202 can store in the memory 308 instructions thatimplement automatic program formatting as described herein. Automaticprogram formatting can be offered to subscribers as an optional featureof the entertainment system 200, in which case, when a subscriber ordersthe entertainment system 200, an associated automatic program formattingcode 316 containing instructions is downloaded to the memory 308 forexecution by the control circuitry 302 within the content receiver 202.If needed, the automatic program formatting code 316 can store videoimage data in memory that resides in a portion of the DVR 306. The DVR306, equipped with high-density memory, is capable of storing largeamounts of data, and is therefore appropriate for storing video imagedata for use by the entertainment system 200.

The content receiver 202 can be connected to the display 206 through aperipheral interface 312 to send and receive signals to and from thedisplay 206. For instance, a television can receive video and audiosignals, whereas a stereo can receive only audio signals. A camcorder,on the other hand, can send video or audio signals to the satellitecontent receiver 202 or receive audio and video signals from the contentreceiver 202 to record. As another example, the peripheral interface 312can include a processor or other electronic components to permit aninterface to content security devices such as an external smart card.The peripheral interface 312 can then encrypt or decrypt content foroutput to other peripheral devices. Thus, the peripheral interface 312can perform one or more functions for multiple peripheral devices,including, but not limited to, the synchronous or asynchronous transferof data between different peripheral devices (e.g., decrypting contentusing a smart card peripheral device and outputting decrypted content toa television at the same time). One skilled in the art will recognizethat the peripheral devices can include many types of commerciallyavailable electronic devices, e.g., a phone line and a modem.

The network interface 310 provides network connectivity, for example, tothe Internet 2106 via a wireless router or a local area network. Theperipheral interface 312 can include components that permit connectionof RJ-45 network cabling and transmission of TCP/IP traffic to otherconnected devices. As another example, a wireless router can be attachedvia the peripheral interface 312 to allow wireless local-area-network(WLAN) data communications using a standard wireless networking protocolsuch as WiMAX, 802.11b or 802.11g. One skilled in the art will recognizethat various other network connections to the content receiver 202 arepossible.

The remote control interface 314 includes instructions that allow theviewer 216 to communicate with the satellite content receiver 202, andcan be implemented using the peripheral interface 312 or by connecting aseparate remote control interface device. The remote control interface314 can translate an input from the user into a format understandable bythe control circuitry 302. The remote control interface 314 can thus beconsidered a user interface (UI). The translation systems can include,but are not limited to, electronic receivers and electronic relays. Theremote control interface 314 can receive commands from more than oneremote control 208. The remote control 208 can use infrared, UHF, orother communications technology. One skilled in the art will recognizethat other means to receive and translate user inputs are possible.

The memory 308 can be used for many purposes, including, but not limitedto, storing programming data, storing commands or functions for thecontrol circuitry 302, storing timers or record events, and storing datafor other devices within or connected to the satellite content receiver202. In one embodiment, the memory 308 provides the long-term storagefunctionality of the content receiver, e.g., for persistent recordings,and may also include operating system software and other data orsoftware necessary to the content receiver, and the DVR 306 providesshort-term storage functionality, e.g., for temporary recordings ofprograms.

The memory 308 can be, for example, an 80 GB Winchester′ hard driveconnected to the control circuitry 302 via a standard IDE/EIDE interfacecable. As another example, the memory 308 may be used to temporarilystore data for processing by the CPU or image processor. One skilled inthe art will recognize that other storage devices and interfaces may besubstituted for those shown and described herein while still remainingwithin the scope of this disclosure. One skilled in the art will alsorecognize that the storage device 308 may include a storage deviceinterface and that portions of the storage device 308 may separately ortogether include an integrated memory, e.g., a memory buffer, commonlyreferred to as cache, and additional processing components or logic. Oneskilled in the art will also recognize that a storage device interfacemay be integrated into the peripheral interface 312. Finally, oneskilled in the art will recognize that the memory 308 may be externaland connected to the content receiver 202. For example, an external harddrive may be connected to the content receiver 202 using USB 2.0 or IEEE1394 FireWire connections. Such an external hard drive may include ascreen for portable viewing of programming stored on it. Furthermore,such a storage device can contain expansion slots, such as IDEconnections, for the provision of additional storage devices to provideadditional capacity at a later time. The design and operation of harddrives and similar devices are well known in the art and need not bedescribed further here.

FIG. 3 shows an overview of information flow 330 through theentertainment system 200 that includes the content receiver 202featuring the automatic program formatting code 316, according to oneembodiment. The information flow 330 starts with original media content332, which undergoes media content analysis 334, to determine when toproduce adapted media content 336. The adapted media content 336 is thendisplayed on the display 206 for the viewer 216.

In one embodiment, the media content 332 is received by the contentreceiver 202 from one of a plurality of sources, e.g., from a TV contentprovider via the satellite dish 210A, for example, through the tuner304, or from the Internet 210B via the network interface 310, or fromstorage in the DVR 306. The media content 332 is in the form ofmultimedia program data that generally includes a sequence of videoframes and accompanying audio data and text data. Individual videoframes are extracted from the media content 332 and are directed to thecontrol circuitry 302, which performs the media content analysis 334.The control circuitry 302 examines the video frames specifically todetermine the aspect ratio of the video data. According to oneembodiment, the control circuitry 302 has previously performed a queryof the display 206 to determine the preferred aspect ratio for a framewhich is shown on the display 206. Accordingly, the aspect ratio of thedisplay 206 is available to the control circuitry 302, and this can becompared to the aspect ratio of the frame of the video data during thestep of media content analysis 334. If the aspect ratio of the frame ofthe media content matches the aspect ratio of the display 206, then noaction is taken, and the frame is passed forward to the display 206 viathe peripheral interface 312. On the other hand, if the frame of themedia content has a different aspect ratio than the video display 206,then the aspect ratio of the frame is modified to match the aspect ratioof the video display 206. This analysis is performed on a particularframe on a frame-by-frame basis of the video as it is received, e.g.,from the tuner 304, for outputting to the display 206 via the peripheryinterface 312. After an initial frame has been analyzed and set to theproper aspect ratio, subsequent steps, need only determine whether ornot the aspect ratio of the current frame matches that of the mostrecent frame. Such an analysis can be performed very quickly withminimal processor overhead.

If the aspect ratio of the present frame matches the aspect ratio of theprevious frame in the sequence, no adjustment is needed. If there is ashift in the aspect ratio, the control circuitry 302 adapts the aspectratio of the current media content 332 to produce the adapted mediacontent 336, in which the program data is changed to have an aspectratio that matches the aspect ratio of the previously displayed videoframe. The program is then displayed with the adjusted aspect ratio sothat when the viewer 216 views the current content on the display 206,the sequence of images will be displayed seamlessly, without a loss ofcontinuity.

Alternatively, the media content analysis 334 can be performed remotelyinstead of being performed locally by the control circuitry 302 on boardthe content receiver 202. For example, after a video frame is extractedfrom the media content 332, associated frame data can be sent to aremote server. The remote server can then perform the media contentanalysis 334 to determine whether or not the aspect ratio needsadjustment. When no adjustment is needed, the server transmits a signalto the content receiver 202 indicating that the original frame issuitable for display on the display 206. When an adjustment is needed,the remote server transmits formatting information to the contentreceiver 202. The content receiver 202 then generates the adapted mediacontent 336 based on the formatting information received from the remoteserver, and displays the adapted video frame on the display 206 via theperipheral interface 312.

With reference to FIG. 4A, for example, media content currently beingshown on the display may have an aspect ratio of 1.33:1, which appearsas a vertically-oriented picture 350 having left and right verticalsidebars 352. New media content that has been received and is beinganalyzed by the control circuitry 302 may have an aspect ratio of 2.4:1,which appears as a horizontally-oriented picture 354 having upper andlower horizontal sidebars 356. When the control circuitry 302 detectsthat the video frames differ, the frame currently under analysis may bemodified to have an aspect ratio of 1.33:1, so it will match the framecurrently being viewed.

Alternatively, every frame can be compared against an independentstandard. For example, all the frames can be modified to have an aspectratio of 1.77:1, which appears as a picture 358 having no sidebars. Thisembodiment is illustrated in FIG. 4B. In this case, the comparison madeby the control circuitry 302 is of the data under analysis against dataresiding in the memory 308, as opposed to comparing the next pictureagainst the current picture. In the example shown, the data representsthe picture 358 having the aspect ratio of 1.77:1. Such an independentstandard may be selected by the viewer from a list of choices, or theindependent standard may be stored as a viewer preference in apreferences file in the content receiver 302.

It is noted that the methods described herein change the actual mediadata to improve the image display, rather than simply changing formatsettings on the display device in an automated fashion. In oneembodiment, in addition to changing the locations of the sidebars, ifneeded, the picture being shown can also be modified to reducedistortion.

FIG. 5 is a flow diagram showing a computer-implemented method 500 ofautomatic program formatting, according to one embodiment. The method500 is codified as automatic formatting instructions 316 to be executedby a microprocessor, e.g., a specialized image processor, within thecontrol circuitry 302 of the content receiver 202 or a remotemicroprocessor. Programmed automatic formatting instructions can bestored on board the microprocessor, or in the computer-readable memory308.

At 502, the content receiver 202 receives the media content 332 in theform of a multi-media stream that includes a video data stream.

At 504, the content receiver 202 extracts a video frame from the videodata stream for analysis, prior to displaying the frame.

At 506, the content receiver 202 determines the location and size ofunused areas of the extracted frame. For example, the content receiver202 determines whether or not vertical or horizontal sidebars arepresent in the image, and if so, how wide the sidebars are. In oneembodiment, such a determination can be made by examining the edges ofthe extracted video frame, starting at the corners, and searching forblack regions that indicate sidebars. For example, pixels can be sampledalong an edge column to detect a full column of black pixels. Then onepixel at the top of each successive column can be examined until anon-black pixel is found. The column of the last black pixel can then besampled to confirm the location of the innermost full column of blackpixels, and hence to deduce the width of a vertical sidebar. A similarscheme can be carried out for rows to recognize and measure the width ofblack regions that are consistent with horizontal sidebars. In thisembodiment, determination of the unused portions of the unused areas ofthe video frame is carried out prior to the frame actually reaching thedisplay 206.

At 508, the content receiver 202 performs a comparison of the unusedareas of the extracted video frame and the unused areas of the framecurrently being displayed. When the unused areas match, the contentreceiver 202 displays the extracted frame without modification and thenreturns to 504 and extracts the next video frame.

At 510, when the unused areas do not match, the content receiver adaptsthe extracted frame to have the same unused areas as the currentlydisplayed frame. That is, wherever sidebars are located on the framecurrently being displayed, corresponding pixels of the new frame willalso show black sidebars.

Adapting the extracted frame to match the displayed frame can beaccomplished in one of several different ways, as illustrated in FIGS.6A-6D. A scene 600 shown in FIG. 6A represents the extracted frame,which, in this example, has no sidebars. In FIG. 6B, the scene 600 isscaled and projected to fill the entire display 206. Scaling can entaileither enlarging or reducing the size of the image. Then, pixels in thesidebar regions are digitally masked by coloring them black. This methodhas the disadvantage that portions of the picture that coincide with thesidebars will be lost. For example, in the scene 600 showing a house, atree, and a sun, the tops of the house and the sun, as well as thebottom of the tree, are masked by horizontal sidebars 602.

Alternatively, in order not to lose portions of the image, the sidebarregions can be defined first and then the picture can be modified to fitthe area between the sidebars. For example, if a low resolution movie isbroadcast for high definition display, the image can be stretched byconverting the received pixel data to the desired resolution so as tofit between the sidebars. As illustrated in FIG. 6C, the horizontalsidebars 602 can be displayed on the display 206 and then the scene 600can be proportionally reduced to a smaller scene 604 that fits betweenthe horizontal sidebars 602. However, because the smaller scene 604 doesnot fill the entire display, there is unused space on the sides as well,which appears as vertical sidebars 606. To eliminate the verticalsidebars, the smaller scene 604 can be stretched in the horizontaldirection only, to fill the unused space on the sides so that verticalsidebars are not needed in addition to the horizontal sidebars 602. Atwo-step process is thus used, which entails first scaling the image andthen stretching the image.

FIGS. 7A and 7B illustrate various ways to stretch the image bydistributing pixel data from the received frame across additional pixelson the display 206. Here, the term pixel expresses the smallest colorunit making up the image, wherein each pixel in a video frame has anassigned numerical value representing a distinct color hue. For example,a two-pixel group 608 of the exemplary smaller scene 604 transitionsfrom a blue sky to a green tree as shown in FIG. 7A. Thus, two adjacentpixels of different colors, blue and green, are to be distributed acrossa larger pixel group 610 (two examples shown, 610A having an odd numberof pixels, and 610B, having an even number of pixels). In both cases,data corresponding to each one of the large pixels in the data stream isdistributed over multiple smaller pixels across the width of the display206. To minimize distortion, when a blue pixel is next to a green pixelin the data stream, and the pair of pixels is to be spread out over thethree-pixel group 610A on the display, a first pixel is shown as blue, asecond pixel adjacent to the blue pixel can be interpolated between theoriginal blue and green hues to be shown as blue-green, and the thirdadjacent pixel is shown as green. Alternatively, the first two adjacentpixels can be shown as blue and the third one as green, or the firstpixel can be shown as blue, and the second and third pixels as green.Using the last two methods, distortion within the image portion of thescreen between the sidebars 602 will be greater than when theinterpolation method is used.

In a second example shown in FIG. 7B, a four-pixel group 610B isavailable on the display 206 for the exemplary pair of pixels in thereceived frame. Thus, the blue pixel can simply be mapped to two bluepixels, and the green pixel can be mapped to two green pixels, whichexpands the image proportionally in the horizontal direction.Alternatively, the two pixels on the ends of the four-pixel group 610Bcan be assigned the original blue and green hues, while the middlepixels can be interpolated to have color values between the originalblue and green hues. Such methods can be used to distribute the videoframe data to fill all of the columns of pixels from left to right whenthe presence of upper and lower sidebars 602 is desired. Likewise,similar methods can be used to distribute the video frame data to fillall of the rows of pixels from top to bottom when the presence of leftand right vertical sidebars 606 is desired.

When there are no sidebars in the currently displayed image, the newframe data is adjusted so that it also has no sidebars. Alternatively,all of the frames can be adjusted to have no sidebars. Thus, if thereceived video frame has horizontal sidebars, these are removed and theimage is distributed over the entire area of the display 206. Or, if thenew video frame has vertical sidebars, these are removed and the imageis distributed over the entire area of the display 206. In general, thedistribution of pixels, as in the example described above, can becarried out by performing a calculation to determine how many highresolution pixels are available to receive the lower resolution data,for example, three display pixels for every pair of received pixels,i.e., 3:2 as in the example above, or twice as many display pixels asreceived pixels, i.e., 2:1, or 4:3, etc. Then, a choice can be made asto whether to map the pixels or to interpolate between the colorspresent in the video data.

Returning to FIG. 5, at 512, the adjusted media content is displayed forthe viewer 216.

At 514, the next video frame is extracted.

At 516, when a change in the aspect ratio is detected in the mediacontent, the content receiver 202 returns to step 506 to perform a newanalysis. Otherwise, the same adaptation is applied at 510 to the nextsuccessive video frames.

As can be appreciated, the present disclosure of analyzing video framesand then matching the video frame to the appropriate display can be usedin a variety of different environments. In a standard TV viewingsituation of the type shown in FIG. 1, the methods as described hereincan be used to adjust a video frame to the proper aspect ratio for anyparticular display, whether the video frame is from an old movie, a VHSformat, an HD format, a Beta format, a Super 8 movie format, an IMAXformat, or any of the various formats which have been used over manyyears. Further, it can be used with any display 206 that a user may havein their home which may be an HD style display, an older CRT display, aflat screen LED, or any of the many displays which are on the markettoday. In addition, the methods as taught herein can be used with manyother types of video displays of the many that are on the market today.For example, the methods as taught herein can be used with a displaythat is a cell phone, a small personal viewing device, a tabletcomputer, e.g., iPAD, a computer display, or any of the many displaysthat are available in electronic devices today. The methods aretherefore flexible across many different platforms.

In a second embodiment, a query can be made of the display itself, andblank regions which show up on the display, rather than analyzing thetype of the display and then analyzing the type of data flow received.Instead, a simple, rather easy test of whether or not the current frameas displayed on the display results in blank pixels is examined, and ifblank pixels are present then an adjustment is made to the aspect ratioof the subsequent frames in order to have few, if any, blank spaces onthe display.

FIG. 8 is a flow diagram illustrating such an embodiment, in which theaspect ratio of video frames is modified as they are shown on thedisplay. According to a method 800, the video display itself is queriedto determine whether or not any pixels on the video display are blank. Ablank pixel on a video display is one in which the video frame beingshown thereon does not have any data within that pixel. In such asituation, the video display has nothing to show, and therefore thatparticular pixel is blank. In a normal screen this will usually show upas a black pixel but may, in other instances, show up as being overlaidby other material which is on the screen, or by another view.

At 802, a video frame of the media content 332 is output from thecontent receiver 202 having the same aspect ratio as the immediatelyprior frame.

At 804, the video frame is displayed on the video display 206.

At 806, the pixels of the video display are analyzed to locate any blankregions of the display.

At 808, a query is made to determine whether or not there are any blankregions in the display. When there are no blank regions in the display,the video format is considered to be currently acceptable, and theprogram returns to the initial step 802 to display, from the contentreceiver 202, media content having the same aspect as the previousframe.

At 810, when the query in step 808 determines that there are blankregions on the display, the aspect ratio of a subsequent frame ismodified to have a new aspect ratio which will reduce the amount ofblank space on the video display. This can be done by adjusting the areaof each pixel, for example, increasing or decreasing the width of eachpixel slightly, or increasing or decreasing the height of the pixels,depending on the location of the blank space. Thus the picture may bestretched or compressed slightly in order to have an acceptable aspectratio to more completely fill the video display 206 according to itsproper aspect ratio.

At 812, the modified video frame is then output, and subsequent stepsare executed as illustrated in FIG. 8.

An advantage of the method 800 is that the image displayed will havelittle or no blank regions or sidebars. However, a disadvantage is thatif a first program is zoomed to eliminate sidebars, then a subsequentprogram that lacks sidebars will be zoomed unnecessarily, and the edgesof the images from the subsequent program may exceed the boundaries ofthe display. Using this method, because the actual frame data is notanalyzed prior to displaying the frame, a change in the format of thereceived media content will not be detectable.

As will be appreciated, in some instances, if all blank spaces in thedisplay are completely filled, the aspect ratio of the video content maybe so distorted as to be unpleasant or awkward to view. In such asituation, it is permitted to change the aspect ratio of the frame toreduce the number of blank spaces shown, but not remove them altogether.Thus, it is within the concepts as taught herein to reduce the blankspaces as shown on the screen, but it is not required that all blankspaces be completely removed, and it is permissible to have a videodisplay in which the aspect ratio of the video frame is changed so thatthe blank spaces are less than what they would be with an unmodifiedframe, but may still result in some edge regions of blank space in thevideo display.

It will be appreciated that, although specific embodiments of thepresent disclosure are described herein for purposes of illustration,various modifications may be made without departing from the spirit andscope of the present disclosure. Accordingly, the present disclosure isnot limited except as by the appended claims.

These and other changes can be made to the embodiments in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificembodiments disclosed in the specification and the claims, but should beconstrued to include all possible embodiments along with the full scopeof equivalents to which such claims are entitled. Accordingly, theclaims are not limited by the disclosure.

The various embodiments described above can be combined to providefurther embodiments. All of the U.S. patents, U.S. patent applicationpublications, U.S. patent applications, foreign patents, foreign patentapplications and non-patent publications referred to in thisspecification and/or listed in the Application Data Sheet areincorporated herein by reference, in their entirety. Aspects of theembodiments can be modified, if necessary to employ concepts of thevarious patents, applications and publications to provide yet furtherembodiments.

1-20. (canceled)
 21. A method, comprising: receiving, by a server at aremote location with respect to a content receiver, aspect ratioinformation associated with a video display; receiving, by the server,video frame data; analyzing, by a microprocessor at the remote location,the video frame data to determine formatting information for use indisplaying an associated video frame on the video display; andtransmitting, by the server, the formatting information to the contentreceiver.
 22. The method of claim 21, wherein the analyzing the videoframe data includes: analyze the video frame data to detect unused areasof an extracted video frame; classifying the video frame data, based onthe analysis, as being in a video format characterized by an initialaspect ratio; and determining whether or not to change the video formatof the video frame data to have an adjusted aspect ratio based on theaspect ratio information associated with the video display.
 23. Themethod of claim 22, wherein the transmitting the formatting informationto the content receiver includes: when it is determined to change thevideo format, including information indicating the adjusted aspect ratioin the formatting information; and transmitting, by the server, theformatting information, including the information indicating theadjusted aspect ratio, to the content receiver.
 24. The method of claim23 wherein the transmitting, by the server, the formatting information,including the information indicating the adjusted aspect ratio, to thecontent receiver causes the content receiver to display on the videodisplay the video frame data having the adjusted aspect ratio.
 25. Themethod of claim 21, further comprising: determining no adjustment isneeded to the video frame data for displaying the associated video frameon the video display; and in response to the determining no adjustmentis needed to the video frame data for displaying the associated videoframe on the video display, transmitting by the server a signal to thecontent receiver indicating that an original frame is suitable fordisplay on the video display.
 26. A non-transitory computer-readablemedium having computer-executable instructions stored thereon that, whenexecuted by at least one computer processor, cause the at least onecomputer processor to perform: receiving, by a server at a remotelocation with respect to a content receiver, aspect ratio informationassociated with a video display; receiving, by the server, video framedata; analyzing, by a microprocessor at the remote location, the videoframe data to determine formatting information for use in displaying anassociated video frame on the video display; and transmitting, by theserver, the formatting information to the content receiver.
 27. Thenon-transitory computer-readable medium of claim 26, wherein theanalyzing the video frame data includes: analyze the video frame data todetect unused areas of an extracted video frame; classifying the videoframe data, based on the analysis, as being in a video formatcharacterized by an initial aspect ratio; and determining whether or notto change the video format of the video frame data to have an adjustedaspect ratio based on the aspect ratio information associated with thevideo display.
 28. The non-transitory computer-readable medium of claim27, wherein the transmitting the formatting information to the contentreceiver includes: when it is determined to change the video format,including information indicating the adjusted aspect ratio in theformatting information; and transmitting, by the server, the formattinginformation, including the information indicating the adjusted aspectratio, to the content receiver.
 29. The non-transitory computer-readablemedium of claim 28 wherein the transmitting, by the server, theformatting information, including the information indicating theadjusted aspect ratio, to the content receiver causes the contentreceiver to display on the video display the video frame data having theadjusted aspect ratio.
 30. The non-transitory computer-readable mediumof claim 26, wherein the computer-executable instructions, when executedby the least one computer processor, further cause the at least onecomputer processor to perform: determining no adjustment is needed tothe video frame data for displaying the associated video frame on thevideo display; and in response to the determining no adjustment isneeded to the video frame data for displaying the associated video frameon the video display, transmitting by the server a signal to the contentreceiver indicating that an original frame is suitable for display onthe video display.
 31. A system comprising: at least one computerprocessor; at least one memory coupled to the at least one processor,the at least one memory having computer-executable instructions storedthereon that, when executed by the least one computer processor, causethe at least one computer processor to perform: receiving, by a serverat a remote location with respect to a content receiver, aspect ratioinformation associated with a video display; receiving, by the server,video frame data; analyzing, by a microprocessor at the remote location,the video frame data to determine formatting information for use indisplaying an associated video frame on the video display; andtransmitting, by the server, the formatting information to the contentreceiver.
 32. The system of claim 31, wherein the analyzing the videoframe data includes: analyze the video frame data to detect unused areasof an extracted video frame; classifying the video frame data, based onthe analysis, as being in a video format characterized by an initialaspect ratio; and determining whether or not to change the video formatof the video frame data to have an adjusted aspect ratio based on theaspect ratio information associated with the video display.
 33. Thesystem of claim 32, wherein the transmitting the formatting informationto the content receiver includes: when it is determined to change thevideo format, including information indicating the adjusted aspect ratioin the formatting information; and transmitting, by the server, theformatting information, including the information indicating theadjusted aspect ratio, to the content receiver.
 34. The system of claim33 wherein the transmitting, by the server, the formatting information,including the information indicating the adjusted aspect ratio, to thecontent receiver causes the content receiver to display on the videodisplay the video frame data having the adjusted aspect ratio.
 35. Thesystem of claim 31, wherein the computer-executable instructions, whenexecuted by the least one computer processor, further cause the at leastone computer processor to perform: determining no adjustment is neededto the video frame data for displaying the associated video frame on thevideo display; and in response to the determining no adjustment isneeded to the video frame data for displaying the associated video frameon the video display, transmitting by the server a signal to the contentreceiver indicating that an original frame is suitable for display onthe video display.